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Hardware can be simplified. 
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The Hardware Represented by This Diagram can be regarded as a unit that can be cascaded to obtain 
as many parallel eigenvector extractors as needed in a given application. The cascaded, identical units 
can be fabricated on a single integrated-circuit chip. 


Dominant-element-based gradient 
descent and dyn amic initial learning 
rate (DOGEDYN) is a method of se- 
quential principal-component analysis 
(PCA) that is well suited for such ap- 
plications as data compression and ex- 
traction of features from sets of data. 
In comparison with a prior method of 
gradient-descent-based sequential 
PCA, this method offers a greater rate 
of learning convergence. Like the 
prior method, DOGEDYN can be im- 
plemented in software. However, the 
main advantage of DOGEDYN over the 
prior method lies in the facts that it re- 
quires less computation and can be im- 
plemented in simpler hardware. It 
should be possible to implement 
DOGEDYN in compact, low-power, 
very-large-scale integrated (VLSI) cir- 
cuitry that could process data in real 
time. 

For the purposes of DOGEDYN, the 
input data are represented as a suc- 
cession of vectors measured at sam- 
pling times t. The objective function 
[the error measure (also called “en- 
ergy” in the art) that one seeks to 
minimize in gradient-descent itera- 
tions] is defined by 
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where m is the number of principal 
components, k is the number of sam- 
pling time intervals (the number of 
measurement vectors), x t is the mea- 
sured vector at time t , and wi is the zth 
principal vector (equivalently, the zth 
eigenvector). The term Ji{wj) in the 
above equation is further expanded by 
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The learning algorithm in DOGEDYN 
involves sequential extraction of the 


principal vectors by means of a gradient 
descent in which only the dominant ele- 
ment is used at each iteration. Omitting 
details of the mathematical derivation 
for the sake of brevity, an iteration in- 
cludes updating of a weight matrix ac- 
cording to 
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where Wy is an element of the weight 
matrix and £ is the dynamic initial 
learning rate, chosen to increase the 
rate of convergence by compensating 
for the energy lost through the previous 
extraction of principal components. 
The value of the dynamic learning rate 
is given by 



where Eq is the energy at the beginning 
of learning and £/_i is the energy of the 
z-lst extracted principal component. 

The figure depicts a hardware archi- 
tecture for implementing DOGEDYN. 
The raw input data, here denoted xj, 


are subtracted from the sum of the 
data previously projected on the previ- 
ous principal components to obtain yj 
(which is equivalent to y t as defined 
above, after appropriate changes in 
subscripts). The X box calculates the 
inner product of vectors y and W[. The 
output of the X box is summed with 
the previously computed product of yj 
and Wy and the result multiplied by the 
dynamic learning rate before updating 

of Wij. 

This work was done by Tuan Duong and 
Vu Duong of Caltech for NASA’s Jet 
Propulsion Laboratory. Further infor- 
mation is contained in a TSP (see page 1 ). 
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